DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 5-7, 10, 12-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Choi et al. (US 2019/0312917 A1, “Choi”) in view of Winterstein et al. (US 2014/0362163 A1, “Winterstein”).
As to claim 1, Choi discloses a method, comprising:
accessing, via a first computing device associated with a first participant at a first physical location, a copresence program configured to support multiple participants, the first physical location including a first physical medium configured to display information of the copresence program (First Co-Presence Collaboration Device 102 associated with collaborator 120 at Site A locally or remotely executes a co-presence collaboration application 106, para. 0010, Fig. 1);
receiving, by one or more processors associated with the first physical location, depth map information of a second participant at a second physical location, the depth map information being derived from a raw image associated with the second participant captured at the second physical location (one or more environmental sensors detects a presence of collaborator 116 at meeting site B, para. 0014; human shapes may be identified from a depth sensor map, para. 0028, 0031);
generating, by the one or more processors associated with the first physical location, a presence shadow corresponding to the second participant, the presence shadow reprojecting aspects of the second participant according to the depth map information (user presence identifier (UPI) 118 is created based on detected user locations and actions, determined via cameras, depth sensors, etc., para. 0026, and may be in the form of a shadow representation, para. 0018, 0026) where the aspects are blurred according to a proximity of each aspect to a second physical medium at the second physical location; and
displaying using the copresence program, on the first physical medium, the presence shadow corresponding to the second participant (UPI 118 is displayed on the wall-mounted touch-screen device 102 at Site A, Fig. 1, para. 0015-0018).
Choi differs from claim 1 in that although it teaches changing the form of the UPI based on a detected distance between the collaborator and the display, e.g. the UPI may grow larger and/or darker as the collaborator gets closer to the display (para. 0017), it does not specifically recite the above underlined limitation, i.e. changing the form by blurring.
Winterstein teaches blurring a user’s image based on distance of the user from the camera device (Fig. 5, para. 0049-0052). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Choi with the above teaching of Winterstein in order to provide an alternative way of indicating distance.
As to claim 2, Choi in view of Winterstein teaches: wherein one or more of the aspects closest to the second physical medium are generated to have a first amount of blurring while one or more of the aspects farthest from the second physical medium are generated to have a second amount of blurring greater than the first amount (Winterstein: mild blurring in closest region, moderate blurring in intermediate region, substantial blurring in farthest region, Fig. 5, para. 0050).
As to claim 3, Choi in view of Winterstein teaches: wherein the presence shadow is configured to illustrate at least one of gesturing or positioning of the second participant (Choi: user presence indicator may be in the form of a shadow representation and may indicate a user’s gesture or location, para. 0014, 0016-0018, 0026, 0037, 0039).
As to claim 5, Choi in view of Winterstein teaches: wherein the presence shadow is only displayed when the second participant is within a threshold distance of the second physical medium (Winterstein: objects beyond a depth threshold are hidden, para. 0044).
As to claim 6, Choi in view of Winterstein teaches:
receiving collaborative information added to the second physical medium (Choi: user input to the whiteboard document, such as by drawing using a stylus or finger-touch, para. 0011); and
displaying the received collaborative information on the first physical medium along with the presence shadow corresponding to the second participant (Choi: edits are made visible, para. 0011, 0024, along with the user presence indicator, para. 0013-0018, 0025).
As to claim 7, Choi in view of Winterstein teaches:
receiving additional collaborative information added to the first physical medium (Choi: co-presence collaboration platform facilitates a shared whiteboard being populated with edits during the course of the conference, para. 0024); and
integrating the additional collaborative information with the received collaborative information to form a set of integrated collaborative information (Choi: para. 0024).
As to claim 10, Choi in view of Winterstein teaches:
the depth map information of the second participant further includes three-dimensional positioning information of a writing implement (Choi: depth sensor data is used to identify coordinates of a user’s hand in three-dimensional space, para. 0040; drawing may be input by finger-touch, para. 0011, which may be displayed as a writing utensil, para. 0037); and
displaying the received collaborative information includes generating a scaled representation of the received collaborative information based on the three-dimensional positioning information of the writing implement (Choi: para. 0011, 0037, 0040).
As to claim 12, Choi in view of Winterstein teaches:
capturing, at the first physical location, a raw image of the first participant (Choi: raw image data, para. 0027); and
transmitting depth map information derived from the raw image of the first participant to a computing device associated with the second physical location (Choi: depth sensor map, para. 0028, 0031).
As to claim 13, Choi in view of Winterstein teaches: wherein a presence shadow of the first participant is not displayed on the first physical medium at the first physical location (user presence indicator may be display exclusively on the display of the remote collaborator, para. 0015).
As to claim 14, Choi in view of Winterstein teaches: receiving audio information of the second participant; and playing the received audio information at the first physical location in conjunction with displaying the presence shadow of the second participant (voice and/or video communications between collaborates at the different meeting sites, para. 0013, 0022).
As to claim 15, Choi in view of Winterstein teaches: using the depth map information to separate images of the second participant at the second physical location from collaborative information added to the second physical medium (Winterstein: presentation, mild and moderate blurring regions are separated based on depth measurements, para. 0042, 0067, Fig. 5).
As to claim 19, Choi in view of Winterstein teaches:
identifying tactile input onto either the first physical medium or the second physical medium (gesture identification determines that a user is pointing to the touchscreen display, para. 0037);
associating the identified tactile input with an action (collaborator action identifier ; and
either (i) changing how selected information is displayed on the first physical medium based on the action, or (ii) performing a computing operation in response to the action.
Claim(s) 4, 11, 16, 22-25 is/are rejected under 35 U.S.C. 103 as being unpatentable over Choi in view of Winterstein, as applied to claim 1 above, and further in view of Chen et al. (US 2015/0244976 A1, “Chen”).
Choi in view of Winterstein differs from claim 4 in that it does not teach: wherein generating the presence shadow includes inverting the depth map information prior to blurring.
Chen teaches displaying a remote user on a collaborative display area (Fig. 4) in which the visualization flips the depth to provide a mirror condition (para. 0026-0028). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Choi in view of Winterstein with the above teaching of Chen in order to emulate a side-by-side collaboration, e.g. mirror image relationship (Chen: para. 0026).
As to claim 11, Choi in view of Winterstein and Chen teaches: wherein the raw image of the second participant is taken from a viewpoint where the second participant is between the second physical medium and a second image capture device at the second physical location (Chen: see Fig. 10).
As to claim 16, Choi in view of Winterstein and Chen teaches: wherein the presence shadow is generated based on a set of skeleton coordinates derived from an image of the second participant (Chen: skeleton data is used to track the positions of the user’s limbs, para. 0041, 0043, 0062-0063).
As to claim 22, Choi in view of Winterstein and Chen teaches a system associated with a first participant at first physical location (see rejection of claim 1 above), the system comprising: a projector configured to display information on a first physical medium located at the first physical location (Chen: projection display implementation, para. 0038, 0053).
As to claim 23, Choi in view of Winterstein and Chen teaches:
receive collaborative information added to the second physical medium (Choi: user input to the whiteboard document, such as by drawing using a stylus or finger-touch, para. 0011); and
cause the projector to display the received collaborative information on the first physical medium along with the presence shadow corresponding to the second participant (Choi: edits are made visible, para. 0011, 0024, along with the user presence indicator, para. 0013-0018, 0025).
As to claim 24, Choi in view of Winterstein and Chen teaches: further comprising an image capture device configured to capture imagery associated with the first participant at the first physical location, the image capture device having a field of view encompassing the first physical medium (Chen: see Fig. 10).
As to claim 25, Choi in view of Winterstein and Chen teaches:
the image capture device is configured to capture a raw image of the first participant;
the one or more processors are configured to generate depth map information of the first participant based on the captured raw image (Choi: raw image data, para. 0027); and
the one or more processors are configured to transmit the depth map information to a computing device associated with the second physical location (Choi: depth sensor map, para. 0028, 0031).
Claim(s) 8, 17-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Choi in view of Winterstein, as applied to claim 7 above, and further in view of Diamant et al. (US 2019/0341050 A1, “Diamant”).
Choi in view of Winterstein differs from claim 8 in that it does not teach: generating one or more bookmarks of the set of integrated collaborative information.
Diamant teaches conference participants indicating a bookmark or point of interest in a conference (para. 0078, 0133). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Choi in view of Winterstein with the above teaching of Diamant in order to easily share or later find relevant portions of the conference.
As to claim 17, Choi in view of Winterstein and Diamant teaches: saving an interactive collaboration of the copresence program in memory (Diamant: audio, video, textual, and/or multi-modal interactive conference is recorded and transcribed, para. 0023).
As to claim 18, Choi in view of Winterstein and Diamant teaches: wherein saving the interactive collaboration includes saving one or more snapshots of collaborative information added to either the first physical medium or the second physical medium (Diamant: transcript event may show a snapshot of the digital whiteboard, para. 0124).
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Choi in view of Winterstein, as applied to claim 6 above, and further in view of Ostap et al. (US 2022/0335620 A1, “Ostap”).
Choi in view of Winterstein differs from claim 9 in that it does not specifically teach: wherein the received collaborative information is displayed at a first resolution while the presence shadow is displayed at a second resolution lower than the first resolution.
Ostap teaches displaying presentation information 910 at a mid-resolution level (para. 0069) and a presenter’s appendage 102 at a low resolution level (para. 0066, Fig. 9B). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Choi in view of Winterstein with the above teaching of Ostap in order to optimize processing performance (Ostap: para. 0064).
Claim(s) 19-21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Choi in view of Winterstein, as applied to claim 1 above, and further in view of Tone (US 2017/0344168 A1).
Choi in view of Winterstein differs from claim 19 in that it does not teach:
identifying tactile input onto either the first physical medium or the second physical medium;
associating the identified tactile input with an action; and
either (i) changing how selected information is displayed on the first physical medium based on the action, or (ii) performing a computing operation in response to the action.
Tone teaches an interactive whiteboard on which a user may select object data by drawing a circle around the object, and selecting an operation button such as COPY button 330 or PASTE button 332 (para. 0093, Figs. 7-10). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Choi in view of Winterstein with the above teaching of Tone in order to provide additional whiteboard functions.
As to claim 20, Choi in view of Winterstein and Tone teaches: wherein changing how the selected information is displayed includes at least one of moving, scaling, translating or rotating a piece of collaborative information (Tone: object data may be dragged and dropped or enlarged, para. 0098).
As to claim 21, Choi in view of Winterstein and Tone teaches: wherein the computing operation is at least one of copying, pasting, saving or transmitting (Tone: object data may be copied, pasted, and transmitted, para. 0055, 0093, 0095-0096).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Gandhi et al. (US 11,245,871) teach a collaborative virtual whiteboard with a silhouette representation of the user (Fig. 9).
Lemay et al. (US 2021/0286502 A1) teach displaying objects as a faint shadow which becomes more concrete and clear as a hand moves closer to the display (para. 0184).
Krans et al. (US 2013/0050398 A1) teach a collaborator becoming more visible as they move closer to the display screen (para. 0061).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Stella L Woo whose telephone number is (571)272-7512. The examiner can normally be reached Monday - Friday, 8 a.m. to 5 p.m.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ahmad Matar can be reached at 571-272-7488. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Stella L. Woo/ Primary Examiner, Art Unit 2693